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ABSTRACT 



A processor presents a sequence of images of a work- 
space that is stretched to enable the user to view a part 
of the workspace in greater detail. The workspace in- 
cludes a middle section and two peripheral sections that 
meet the middle section on opposite edges. Each of the 
sections appears to be a rectangular two-dimensional 
surface and they are perceptible in three dimensions. 
When the user is viewing the middle section as if it were 
parallel to the display screen surface, each peripheral 
section appears to extend away from the user at an angle 
from the edge of the middle section so that the periph- 
eral sections occupy relatively little of the screen. When 
the user requests stretching, the middle section is 
stretched and the peripheral sections are compressed to 
accommodate the stretching. When the user requests 
destretching, the middle section is destretched and the 
peripheral sections are decompressed accordingly. The 
workspace includes display objects that may cross the 
edges from the middle section to the peripheral sections 
during stretching. Stretching and destretching are per- 
formed in small steps to preserve object constancy for 
display objects that cross the edges. The display objects 
can 1 have fixed sizes, in which case each display object 
is perceived as being tacked to the workspace at a point. 
Or the display objects can stretch and destretch with 
.-the workspace, in which case additional internal detail 
may become visible as a result of stretching and each 
display object is perceived as being glued to the work- 
space over its entire extent. Additional display objects 
can become visible as a result of stretching. 



22 Claims, 10 Drawing Sheets 
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Fig. 1A 
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Fig. 2C 
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FIG. 4 for a tree structured text file. Section 4 also 
OPERATING A PROCESSOR TO DISPLAY describes fisheye views for botanical taxonomies, legal 

STRETCHED CONTINUATION OF A codes, text outlines, a decisions tree, a telephone area 

WORKSPACE code directory, a corporate directory, and UNIX file 

5 hierarchy listings. Section 5 indicates that a display- 
This is a continuation of application Ser. No. relevant notion of a priori importance can be defined for 
07/488,587, filed Mar. 5, 1990, now abandoned. lists, trees, acyclic directed graphs, general graphs, and 

BACKGROUND OF THE INVENTION Euchde^n spaces; unlike the geographic .sample which 

inspired the metaphor of the "New Yorker's View," the 
The present invention relates to techniques for dis- 10 underlying structures need not be spatial, nor need the 
play presentation of a workspace within which display output be graphic. FIG. 6 shows a fisheye calendar, 
features are perceived as having positions relative to Fairchild, K. M., Poltrock, S. E., and Furnas, G. W., 
each other. "SemNet: Three-Dimensional Graphic Representations 

Spence, R. and Apperley, M., "Data Base Naviga- 0 f Large Knowledge Bases," in Guindon, R., Ed., Cog- 
tion: An Office Environment for the Professional," 15 nitiye Science and its Application for Human Computer 
Behaviour and Information Technology, Vol. 1, No. 1, Interaction, Lawrence Erlbaum, Hillsdale, NJ„ 1988, 

1982, pp. 43-54, describe bifocal viewing systems. As pp 2 01-233, describe SemNet, a three-dimensional 
described in relation to FIG. 5(c), a surface on which graphical interface. SemNet presents views that allow 
icons are located, referred to as the wall, can be pres- uscrs to cxammc local detail while maintaining a global 
ented in three separate viewports; the central region 20 representation of tne rest of ^ t Pledge base. As 
presents a part of the wall in sufficient detail to permit shown ^ described m relation to M SemNet 
the user to read itscontents in full; fcvo outer regions resents a base as a directed graph in a 
retain less detail. T*e user can scroll by touching an ^^^5^ s ace , ^ elements of knowiedge 
item in one of the outer regions and pulling it into the M labded rectangles connected by lines or 
central region. In response, the whole strip of data is 25 ^ qu * describes ^ appli . 
moved across the screen, preserving the spattal relation- ^ * §e lg 

ships between items. The ^^^f^J^^ is shown and described in relation to FIG 4-1. As de- 
the outer regions should not merely be a demagrnfied ^ . determine 

version ofthe central region represenUtion, but rather a degr ee^f.Lerest value for aU the objects, including 

representation more appropriate to their lower resolu- 30 " ^ ^ ^ " v<uuc lu * , !: J , ' . 7Z 
tion. As described in relation to FIGS. 6 and 7, scrolling ' luste K r object mat represent a set of elements. One 
can be in vertical and horizontal dimensions. As de- **wback of this is that when tfie viewpoin is moved, 
scribed in relation to FIGS. 8 and 9, zooming can pro- * c displayed change abruptly between 

vide additional levels of information. a of elemen * and a cluster object at invisible 

In a video presentation made publicly available in 35 subdivision boundaries. This can be ameliorated by 

1983, Spence illustrated a bifocal display like that de- ™ m & euchdean distance to tell when a boundary is 
scribed in the article by Spence and Apperley. The bem * approached, then highh^ting the clusters that 
video shows a column from a newspaper extending are about to change. Pages 220-221 describe how three- 
across the space between two pegs and, outside the dimensional point perspective achieves a fisheye effect, 
pegs, extending backward at an angle away from the 40 ™& changing in size based on importance; 
viewer, so that the part ofthe column between the pegs the ob J ects could also chm & m fom, from an icon to 
is legible while the parts outside the pegs are visible but smaller boxes and finally to a point Section 5.2.4, on: 
illegible. The video also shows a strip of paper with Pages 227-228, describes hyperspace movement, in 
colored rectangles attached to it, again with a fully which toe nodes connected to a selected knowledge 
visible part between the two pegs and the two parts 45 element are temporarily moved to positions around it, 
outside the pegs extending backward at an angle so that and then snap back to their original positions after a 
the colored rectangles appear as narrow colored strips. new node * s selected. 

Finally, the video shows a similar strip of paper in SUMMARY OF THE INVENTION 

which the colored rectangle between the pegs includes 

text that is legible, while the colored rectangles on the 50 Th e present invention provides techniques for pres- 
parts that extend backward each have a single large cnting a workspace with stretching to facilitate view- 
character, resembling the features in FIG. 5(c) of the tog. In response to a user request for stretching, the 
article by Spence and Apperley. objects in the workspace are presented in a manner 

Furnas, G. W., "Generalized Fisheye Views," CHI indicating that the workspace is stretched. The user can 
'86 Proceedings, ACM, April 1986, pp. 16-23, describes 55 also request destretching, in which case the objects are 
fisheye views that provide a balance of local detail and again presented as if the workspace were not stretched, 
global context Section 1 discusses fisheye lenses that One aspect of the invention is based on the recogni- 
show places nearby in great detail while still showing tion of problems with conventional techniques for pres- 
the whole world, showing remote regions in succes- enting a more detailed view of a part of a workspace, 
sively less detail; a caricature is the poster of the "New 60 Some conventional techniques provide more detail 
Yorker's View of the United States." Section 3 de- by sacrificing the user's perception that the displayed 
scribes a degree of interest (DOT) function that assigns, features have relative positions, thereby sacrificing the 
to each point in a structure, a number telling how inter- perception of the workspace. In other words, the fea- 
ested the user is in seeing that point, given the current tares shown at greater detail and the features shown at 
task. A display can then be made by showing the most 65 less detail are presented in such a way that the user has 
interesting points, as indicated by the DOI function. difficulty perceiving them as being positioned in rela- 
The fisheye view can achieve, for example, a logarith- tion to each other in a workspace. For example, the less 
mically compressed display, of a tree, as illustrated by detailed features may be presented as abstracted objects 
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that appear to be outside the space in which the more FIGS. 2A, 2B, and 2C are views of workspaces, each 

detailed features appear. of which has three sections, with the workspaces of 

Other conventional techniques such as magnifying or FIGS. 28 and 2C perceptible as stretched continuations 

zooming, provide greater detail be temporarily chang- of the workspace of FIG. 2A according to the inven- 

ing the scale at which part of the workspace is pres- 5 tion. 

ented. But changing the scale of a part of the workspace FIG. 3 is a plan view of a workspace like those in 

can confuse the user by eliminating cues about context: FIGS. 1A and 2A, showing regions. 

If a part of the workspace is presented at increased FIG. 4 is a flow chart showing general steps in pres- 

scale, the surrounding area may no longer be presented enting workspaces like those in FIGS. 1A and IB. 

because of insufficient display area; or the surrounding 10 FIG. 5 is a plan view of a workspace like that in FIG. 

area may have a discontinuous border with the in- 3, illustrating how a viewing position within a three-di- 

creased scale part. These techniques therefore deprive mensional space can be used to determine the sequence 

the user of important cues about the relation between in which the panels are drawn, 

the increased scale part and the surrounding area. FIG. 6 is a sequence of two plan views of workspaces 

This aspect of the invention is further based on the 15 like those in FIG. 3, showing how scrolling can be 

discovery that these problems can be solved by present- performed. 

ing the workspace as if part of it were distorted, such as FIG. 7 is a flow chart showing steps in an animation 

by stretching. The user can then view the stretched part loop that performs stretching, shrinking, and scrolling 

of the workspace in greater detail without losing the of workspaces like those in FIGS. 1A and IB. 

perception of its position relative to the surrounding 20 FIG. 8 is a block diagram showing components of a 

area. For example, if a number of objects are presented system that can perform the steps in FIG. 7. 

on the workspace, stretching reduces the density of the _ ATT ™ r^cr-p fpttam 

objects in the stretched area and therefore assists the DETAILED DESCRIPTION 

user in locating an object of interest The objects them- A. Conceptual Framework 

selves need not be modified in scale or shape. If the The following conceptual framework is helpful in 

objects are, however, modified in scale or shape, they understanding the broad scope of the invention, and the 

can be presented with additional internal detail, and terms defined below have the meanings indicated 

previously invisible objects can become visible at a throughout this application, including the claims. This 

larger scale or an extended shape. concc Ptual framework is a modification and extension 

A closely related aspect of the invention is based on of those set forth in the following copending, coas- 

the recognition that when one part of the workspace is signed U.S. patent applications, which are incorporated 

stretched, other parts must be compressed or otherwise herein by reference: U.S. patent application Ser. No. 

modified to accommodate the stretching. This problem 07/030,766, issued as U.S. Pat No. 5,072,412, entitled 

can be solved by presenting the workspace divided into 35 "User Interface with Multiple Workspaces for Sharing 

panels or sections— when part of the workspace in one Display System Objects"; U.S. patent application Ser. 

section stretches, other sections are modified to accom- No. 07/241,525, issued as U.S. Pat No. 5,107,443, enti- 

modate the stretching. tied "Private Regions Within a Shared Workspace"; 

For example, the workspace can be presented as a and U.S. patent application Ser. No. 07/459,010, contin- 

series of three panels or. sections, each of which is a ued as application Ser. No. 07/795,238 which is in turn 

two-dimensional surface. The middle section, presented continued as application Ser. No. 08/066,311, entitled 

as parallel to tbie display screen surface, can be stretched "Display of Hierarchical Three-Dimensional Structures 

as requested by the user. Each peripheral section can be with Rotating Substructures." 

presented as extending away from the user at an angle A "data processing system" is a system that processes 
from an edge of the middle section, so that each periph- 45 data, A "data processor" or "processor" is any compo- 
eral section occupies relatively little screen space. nent or system that can process data, and may include 
When the middle section stretches, objects on each one or more central processing units or other process- 
peripheral section can be moved away from the user, in ing components. A "user input device" is a device such 
effect compressing each peripheral section to accom- as a keyboard or a mouse that can provide signals based 
modate the stretching. On a large display, this technique 50 on actions of a user. An "image output device" is a 
makes good use of human peripheral vision. device that can provide an image as output. A "display" 
Another closely related aspect of the invention is is an image output device that provides information in 
based on the recognition of a problem in moving objects visible form, such as on the screen of a cathode ray tube, 
across the edge between the middle section and one of The visible pattern presented by a display is an "image." 
the peripheral sections. Object constancy may be lost if 55 A wide variety of display techniques for data process- 
an object is inappropriately presented as it crosses the ing systems are available including, for example, various 
edge during stretching. This problem can be solved by graphical user interfaces, but, despite their diversity, 
performing the stretching operation in steps that are these techniques tend to have certain common charac- 
small enough that object constancy is preserved. teristics. One fundamental common characteristic is 
The following description, the drawings and the 60 that a display produces human perceptions. In this ap- 
claims farther set forth these and other objects, features, plication, the term "display feature" refers to any 
and advantages of the invention. human perception produced by a display. 

^ „ ^ WTT , ^ . ™,^r^ A "workspace" is a display feature within which 

BRIEF DESCRIPTION OF THE DRAWINGS other d^iay features appear to have positions relative 

FIGS. 1A and IB are views of workspaces, each of 65 to each other. In other words, a workspace is presented 

which has three sections, with the workspace of FIG. when the perception is produced that the other display 

IB perceptible as a stretched continuation of the work- features have relative positions. The workspace is the 

space of FIG. 1A according to the invention. perceived space within which the relative positions 
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exist A workspace can have visible boundaries on the 
display, but not all workspaces have visible boundaries. 

The present invention is particularly applicable to 
workspaces that include one or more sections, each of 
which is perceptible as a "surface," meaning a two-di- 5 
mensional locus of points in three-dimensional space, 
the locus having a height z above each point of a contig- 
uous region of a coordinate plane that is specified by a 
function f(x,y) that is continuous over the region. For a 
"two-dimensional surface," fi(x,y)=Z, a constant, over 10 
the region, so that a surface that is perceived as a contig- 
uous part of a plane is a two-dimensional surface. 

Display features such as sections of a workspace are 
"perceptible in three-dimensions" when they are per- 
ceptible as having relative positions in a space that has 15 
three orthogonal dimensions. Typically, a display will 
have a two-dimensional display surface and the percep- 
tion of the third dimension will be produced with visual 
cues such as obscuring of distant features by near fea- 
tures, size changes in features that change distance, and 20 
different shading of features at different distances. In 
effect, two-dimensional sections of a first workspace 
can be presented within a second workspace that is 
three-dimensional. 

A second display feature is perceptible as a "continu- 25 
ation" of a first display feature when the presentation of 
the second display feature follows the presentation of 
the first display feature in such a way that the user 
perceives the first display feature as being continued 
when the second display feature is presented. This can 30 
occur, for example, when the successive display of two 
display features is so close in time and space that they 
appear to be the same display feature. This perception 
may be accompanied by the perception that the first 
display feature has moved. An example of this is the 35 
phenomenon called "object constancy." 

If a second workspace is perceptible as a continuation 
of a first workspace, it may also be perceptible as a 
"modified continuation" if presentation of the second 
workspace produces the perception that the first work- 40 
space has been modified. More specifically, if a second 
workspace is perceptible as a modified continuation of a 
first workspace, it may also be perceptible as a "dis- 
torted continuation" if presentation of the second work- 
space produces the perception that the first workspace 45 
has been distorted, resulting in a non-uniform change in 
the relative positions of display features within it Fur- 
ther, a part of the second workspace is perceptible as a 
"stretched continuation" of a part of the first workspace 
if it produces the perception that the part of the first 50 
workspace has been stretched. The second workspace is 
perceptible as a "stretched continuation" of a first 
workspace if it includes a part that is a stretched contin- 
uation of part of the first workspace and if its other parts 
have been modified to accommodate the stretching. On 55 
the other hand, a second workspace may be a modified 
continuation of a first workspace without being dis- 
torted, such as when the first workspace is increased or 
decreased in size, either through a magnification or 
demagnification or by increasing or decreasing its ex- 60 
tent 

A "display object" is a display feature that has per- 
ceptible extent and that is coherent in the sense that it is 
perceived as a continuing bounded unity. Familiar ex- 
amples of display objects include characters, icons, and 65 
windows. A display object within a workspace may not 
be modified when the workspace is distorted, in which 
case it may be perceived as being 'Hacked" to the work- 
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space at a point On the other hand, if the display object 
is distorted with the workspace, it may be perceived as 
being "glued" to the workspace over its entire extent. 

A "data structure" is a combination of interrelated 
items of data. A processor "accesses" a data structure 
by any operation that retrieves or modifies data in- 
cluded in the data structure, such as by reading or writ- 
ing data at a location in memory that is included in the 
data structure. A processor can be "connected for ac- 
cessing" a data structure by any combination of connec- 
tions with local or remote memory or input/output 
devices that permits the processor to access the data 
structure. 
B. General Features 

FIGS. 1A-3 illustrate general features of the inven- 
tion. FIGS. 1A and IB each show a workspace with 
three sections, the workspace of FIG. IB being percep- 
tible as a stretched continuation of the workspace of 
FIG. 1A. FIGS. 2A, 2B, and 2C each show a work- 
space with three sections, the workspaces of FIGS. 2B 
and 2C being perceptible as stretched continuations of 
the workspace of FIG. 2A. FIG. 3 shows how a work- 
space like those in FIGS. 1 A and 2A can be divided into 
regions. 

Workspace 10 in FIG. 1A includes a number of nodes 
including nodes 12, 14, 16, and 18, each shown as a 
card-like display object that has a four-sided geometric 
shape. These nodes are perceptible as having positions 
relative to each other, giving rise to the perception of 
workspace 10 as a space within which the nodes have 
relative positions. The positions of the nodes differ in 
two dimensions, which can be treated as x and y coordi- 
nates. For example, the x coordinate could be indicate 
linear differences in time and the y coordinate could 
indicate difference in nature or quality of nodes. Where 
more than one node is at the same coordinates, the 
coordinates can be offset to make all of the nodes visi- 
ble, even though the upper nodes obscure parts of the 
lower nodes. 

A number of static cues contribute to the perception 
that the nodes have relative positions in a space: The 
shading of the nodes and the background on which they 
are shown is such that the nodes appear to be on or in 
front of the background; some of the nodes, such as 
node 18, obscure parts of other nodes, suggesting that 
they are on or in front of the other nodes; and work- 
space 10 includes three sections, middle section 20 and 
peripheral sections 22 and 24. Peripheral sections 22 and 
24 are perceptible as connected to the left and right 
edges of middle section 20 and as extending at an angle 
away from the observer so that nodes in peripheral 
sections 22 and 24 are in workspace 10 with nodes in 
middle section 20. 

In addition, other cues can contribute to the percep- 
tion that the nodes have relative positions in space: In 
response to a request from a user, a modified continua- 
tion of workspace 10 can be presented in which one or 
more nodes have moved to different positions or have 
moved across one of the edges from one section to 
another, while the relative positions of the nodes are 
unchanged. Also, a user may perceive the nodes as 
having relative positions in space based on past experi- 
ence with similar images. 

Workspace 10 is also perceptible as being in a three- 
dimensional workspace, the cues that contribute to this 
perception include the difference in shading between 
middle section 20 and peripheral sections 22 and 24; the 
difference in shape of the nodes in middle section 20 and 



03/30/2004, EAST Version: 1.4.1 



5,339,: 

7 

peripheral sections 22 and 24; and the tapered shapes of 
peripheral sections 22 and 24, which suggest that they 
extend, in a three-dimensional workspace, away from 
the user viewing workspace 10. 

Workspace 40 in FIG. IB can be perceived as a 5 
stretched continuation of workspace 10 if presented 
appropriately. For example, if workspace 40 is pres- 
ented immediately after workspace 10, the user may 
perceive it as a stretched continuation. Or a sequence of 
images can be presented beginning with workspace 10 10 
and continuing with several workspaces, each of which 
appears to be a stretched continuation of workspace 10, 
including workspace 40. 

If workspace 40 is perceived as a stretched continua- 
tion of workspace 10, then nodes 42, 44, 46, and AS are 15 
perceived as continuations of nodes 12, 14, 16, and 18, 
respectively. Similarly, middle section 50 is perceived 
as a continuation of middle section 20, and peripheral 
sections 52 and 54 are perceived as continuations of 
peripheral sections 22 and 24, respectively. 20 

Workspace 40 is perceptible as having been stretched 
in both directions from fixed point 60 at the center of 
middle section 50, as indicated by arrows 72 and 74. 
Fixed points 62 and 64 at the outer ends of peripheral 
sections 52 and 54 and arrows 76 and 78 indicate that 25 
peripheral sections 52 and 54 are perceptible as having 
been compressed to accommodate the stretching of 
middle section 50. As result of the stretching, nodes 42 
and 46 are perceptible as having crossed the edges onto 
peripheral sections 52 and 54, respectively, while the 30 
nodes under node 48 are less obscured than the nodes 
under node 18 in workspace 10, making more internal 
detail visible. 

Nodes in workspace 40 are perceptible as being 
tacked to a point in workspace 40 rather than being 35 
glued to workspace 40. As shown, the nodes themselves 
are not stretched, but each is shown as if its upper left 
hand corner were tacked to a position in workspace 40, 
with the position being displaced during stretching. 

In contrast, nodes in the workspaces in FIGS. 2A, 40 
2B, and 2C are bar-like display objects that are percepti- 
ble as being glued in place, so that they stretch and 
destretch with the workspaces. Workspace 80 in FIG. 
2A includes nodes 82 and 84, for example, while work- 
space 88 includes nodes 90 and 92 that are perceptible as 45 
continuations of nodes 82 and 84, respectively. As can 
be seen, node 90 is a stretched continuation of node 82, 
allowing for presentation of additional internal detail. 
Node 92, however, is a compressed continuation of 
node 84. 50 

Workspace 94 in FIG. 2C is like workspace 88 except 
that it also includes node 96, a node that was invisible in 
workspace 80 but that has become visible due to the 
change in size and shape resulting from stretching. This 
is another way in which stretching may make additional 55 
details visible to the user. 

The choice between nodes that are tacked and nodes 
that are glued depends on the nature of the presentation 
being made. Gluing is likely to be more appropriate for 
presentations that involve a time scale or dependency, 60 
such as Gantt Charts, Pert Charts and the like. Gluing 
may also be appropriate for certain bar charts, pie 
charts, and other presentations with a strong geometri- 
cal component 

FIG. 3 illustrates how a workspace can be divided 65 
into regions for purposes of positioning nodes or other 
display features in the workspace during stretching. 
Workspace 100, like workspaces 10 and 80, includes 
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middle section 102 and peripheral sections 104 and 106, 
seen in profile as if viewed from above in a three-dimen- 
sional workspace within which workspace 100 is posi- 
tioned. 

Workspace 110 thus extends from left end 110 of 
peripheral section 104 to edge 112 between peripheral 
section 104 and middle section 102, thence to edge 114 
between middle section 102 and peripheral section 106, 
and thence to right end 116 of peripheral section 106. 
One of the cues that produces the perception of a three- 
dimensional workspace is that peripheral sections 104 
and 106 are presented as if they meet middle section 102 
at angles, with the angle 6 at edge 114 being shown as 
approximately 135 degrees. In this perceived three-di- 
mensional workspace, peripheral section 104 has a 
width of LW, central section 102 has a width of W, and 
peripheral section 106 has a width of RW. LW, W, and 
RW can be held constant during stretching, as in the 
implementation discussed below, or could be adjusted 
in an appropriate way. 

Middle section 102, as shown, has a center point 120, 
and is divided at points 122 and 124 into a central region 
and two side regions. The points 122 and 124 are posi- 
tioned at a distance A from the edges 112 and 114, re- 
spectively, so that the width of each side region is A and 
the width of the central region is W— 2 A. The side 
regions are, in effect, moved from middle section 102 to 
peripheral sections 104 and 106 during stretching. In 
other words, the central region of middle section 102 is 
stretched until it fills the area of middle section 102, 
while each side region and the adjacent peripheral sec- 
tion are compressed until they fit into the area of the 
peripheral section. 

To perform a sequence of steps of stretching or de- 
stretching, the width of each side region, A, can vary in 
steps between A=0 when the workspace is unstretched 
and A=W/2.5 when the workspace is fully stretched. 
The steps between values of A are sufficiently small that 
the sequence of steps of stretching or destretching pro- 
duces the perception of smooth movement of display 
features across die edges between sections, preserving 
the object constancy of display objects. 

C. Implementation 

The general features described in relation to FIGS. 
1A-3 could be implemented in many ways on a wide 
variety of data processing systems. FIGS. 4-8 illustrate 
features of a current implementation, which runs on a 
Silicon Graphics workstation. This implementation is 
called the Perspective Wall, and the Perspective Wall 
has three sections, called panels. The Perspective Wall 
has been implemented for a Unix file browser: The 
horizontal axis represents time, specifically the last 
modification time for each file. The vertical axis repre- 
sents file type, such as Lisp, C, Object, PDL, backups, 
or other. The nodes are files, and a rectangular card-like 
display object for each node is positioned on the wall 
surface according to its time and file type. If multiple 
nodes have the same time and file type, they are over- 
lapped as if stacked vertically. Color depth cues are 
used on the side panels to enhance the perception of 
three dimensions. Similarly, the visual cues are pro- 
vided so that the user perceives a floor plane supporting 
the Perspective Wall, further enhancing the perception 
of three dimensions. 

1. Presentation 

FIG. 4 shows general steps in presenting the Perspec- 
tive Wall. In the step in box 150, the current viewing 
position is received; this step can also include receiving 
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any other data needed to perform the steps in FIG. 4, that of the edge between middle panel 204 and right side 

such as a handle for the data structure that includes data panel 208 but its z coordinate is greater than that of 

for each of the nodes presented on the Perspective middle panel 204; for viewing positions in region 214, 

Wall. This data structure can include, for each node, a right side panel 208 is drawn first, followed by left side 

unique identifier (UID); a position on the unstretched 5 panel 206 and middle panel 204. 

Perspective Wall expressed in x and y coordinates; a The technique of FIG. 5 for determining the order in 

position on the stretched Perspective Wall expressed as which the panels are drawn may be advantageous in 

an x coordinate; and a text that can, for example, be some implementations because it avoids the need for 

represented in the node's display object Z-buffer hardware. Also, the nodes* display objects can 

The step in box 152 in FIG. 4 tests whether a stored 10 be presented so that they are visible from either side of 

value A is equal to zero. If not, the Perspective Wall is the panels, but if Z-buffer hardware is available, each 

stretched, so that the step in box 160 begins an iterative node could instead be positioned just in front of its panel 

loop that goes through the nodes, determining the posi- so that it would only be visible from the front of the 

tion of each node on the stretched Perspective Wall. panel. 

The data for each node can be accessed in the data 15 When the panel sequence has been determined, the 

structure using the nodes UID, which can for example step in box 172 begins an outer iterative loop that draws 

be numbers, so that the nodes are accessed in the itera- the panels in the determined sequence. The step in box 

tive loop in the order of their UIDs. 174 draws the next panel in the sequence, rotating it 

The step in box 162 accesses and used the x coordi- about the appropriate axis in the three-dimensional 

nate on the unstretched Perspective Wall of the node 20 workspace 200 to provide the angle 6 as discussed 

being handled to determined which of the regions above in relation to FIG. 3. To facilitate the perception 

shown in FIG. 3 includes it. The step in box 164 then of three dimensions, Gouraud shading can be used to 

applies the appropriate equation for that region to ob- determine the shading of the panels in this step, with the 

tain the x coordinate on the stretched Perspective Wall, panels being more darkly shaded as they are more dis- 

which could then be stored in the data structure for 25 tant from the viewing position, 

subsequent use. The step in box 176 begins an inner iterative loop 

Equations that can be applied in box 164 are as fol- that, during the drawing of each panel, goes through 

lows, where x indicates the unstretched position, x' the nodes again to find and handle the nodes in the panel 

indicates the stretched position, and LW, W, RW, and being drawn. The test in box 180 determines, for the 

A indicate the widths discussed in relation to FIG. 3; (a) 30 next node, whether its display object is entirely on the 

For a node on the panel corresponding to peripheral panel being drawn, which can be determined from the 

section 104 or for a node on the left side region of the node's x coordinate and the length of its display object, 

panel corresponding to middle section 102 in FIG. 3, i.e. The x coordinate in the unstretched Perspective Wall is 

0<x<LW+A: x' fl =Xfi- A(xo/(LW-h A)), a where used in box 180 if A=0; otherwise, the stretched x coor- 

Xa=x and x<r'=x\ (b) For a node on the central region 35 dinate from box 166 is used. 

of the panel corresponding to middle section 102 in If the node's display object is entirely on the panel 

FIG. 3, i.e. LW+A^x<LW+W- A: being drawn, the step in box 182 includes the node on 

x'$= xa— A((W/2 — xi)/(W/2 — A)), where x*-x-LW the panel. If the node's display object is not entirely on 

andx , «X£, , -fLWsothatx*<W/2onthelefthalfofthe the panel being drawn, the step in box 184 clips the 

central region and x*> W/2 on the right half; it may 40 node's display object, cutting off the part that is not on 

also be necessary to apply this or another equation to the panel being drawn, which may be the entire display 

nodes on the left and right side regions of middle section object. Then, in box 186, what is left of the display 

102 so that clipping can be performed to eliminate nodes object after clipping, if anything, is included in the panel 

in the side regions, (c) For a node on the right side being drawn. Shading techniques could be applied to 

region of the panel corresponding to middle section 102 45 the node display objects, 

or on the panel corresponding to peripheral section 106 2. Scrolling 

in FIG. 3, i.e. LW+W-A^x<LW+W+RW; When the Perspective Wall has been presented, the 

x' e =Xc-f A((RW— Xc)/(RW-f A)), where user can request a scrolling operation to bring a selected 

x c =x— LW—W and x'=Xc'+LW+W. node to a primary viewing position at the center of the 

When all the nodes have been handled in this manner, 50 middle section of the Wall, as illustrated in FIG. 6. Each 

the step in box 170 uses the viewing position received in of the node display objects can be a selectable unit so 

box 150 to determine the sequence in which the panels that the user can request the scrolling operation simply 

of the Perspective Wall are drawn. FIG. 5 shows one by selecting a node's display object with a user input 

way in which this sequence can be determined within device such as a mouse. Picking hardware can be used 

three-dimensional workspace 200 that includes a Per- 55 to determine which display object has been selected, 

spective Wall with middle panel 204 and side panels 206 FIG. 6 shows wall 230 with middle panel 232 and side 

and 208; A viewing position is in region 210 if its x panels 234 and 236. Side panels 234 and 236 are shown 

coordinate in workspace 200 is greater than that of the as being of approximately equal length, 

edge between middle panel 204 and right side panel 208; Selected position 238 is positioned in the display ob- 

for viewing positions in region 210, left side panel 206 is 60 ject of a node on side panel 234, near the edge at which 

draw first, followed by middle panel 204 and right side it meets middle panel 234. 

panel 208. A viewing position is in region 212 if its x In response to the user input signal selecting position 
coordinate is less than that of the edge between middle 238, the display object of the selected node is high- 
panel 204 and right side panel 208 but its z coordinate is lighted in blue, those of related nodes such as other files 
less than that of middle panel 204; for viewing positions 65 in the same directory are presented in white, and display 
in region 212; middle panel 204 is drawn first, followed objects of other nodes are presented in grey. Text asso- 
by right side panel 208 and left side panel 206. A view- ciated with the selected node and related nodes is also 
ing position is in region 214 if its x coordinate is less than displayed in their display objects. Then, after a scrolling 
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operation, wall 250 is presented with middle panel 252 
and side panels 254 and 256. Because of the scrolling 
operation, side panels 254 and 256 are not of equal 
length, but side panel 254 has been shortened and side 
panel 256 has been lengthened as necessary, to bring the 5 
selected position 258 (corresponding to position 238) to 
the center of middle panel. 252, at which it is in the 
primary viewing position. 

To maintain object constancy during selection scroll- 
ing, a sequence of walls can be presented between wall 10 
230 and wall 250 using animation techniques. A gover- 
nor is used to determine how many animation cycles are 
necessary to accomplish selection scrolling in a prede- 
termined fixed time and to determine a step length by 
dividing the total distance from the selected position to 15 
the primary viewing position by the number of anima- 
tion cycles. The distance from the selected position to 
the primary viewing position can be determined from 
the node's x coordinate on the wall and the lengths of 
the left side panel 234 and middle panel 232. Each ani- 20 
mation cycle that occurs during the selection scrolling 
operation then moves the wall by the step length from 
the previous position. 

Continuing scrolling can be provided in a manner 
similar to selection scrolling. The user could request 25 
continuing scrolling by depressing a mouse button 
when the pointer is indicating a location on the wall that 
is not in a node display object If the indicated location 
on the wall is left of the center point, the wall scrolls to 
the right, and if the indicated location is right of the 30 
center point, the wall scrolls to the left. The length of 
each continuing scrolling step can depend on how far 
the indicated location is from the center point, with the 
step length being longer if the indicated location is 
farther from the center point. Continuing scrolling can 35 
terminate when one of the ends of the wall reaches the 
center point. 

3. Animation 

FIG. 7 shows steps in an animation loop that can 
perform stretching, shrinking, selection scrolling, and 40 
continuing scrolling. One reason for using animation for 
these operations is to shift some of the cognitive pro- 
cessing load to perceptual activity by making use of the 
object constancy property of the human perceptual 
system. In this way, large scale changes in the Perspec- 45 
tive Wall can be made without the cognitive reassimila- 
tion that would be necessary without animation. Ani- 
mation allows the perceptual system to follow the 
changes while they are being made, so that when the 
final form is displayed the user is already aware of what 50 
is being seen. 

The step in box 270 determines whether a user re- 
quest has been received from a user input device since 
the last animation loop. If so, the step in box 272 
branches based on the user request 55 

If the user is depressing a key that requests a stretch 
operation, the step in box 274 tests whether the stored 
value A is already at the maximum value, which may be 
W/2 as discussed above. If not A is increased by one 
incremental amount in box 276 before presenting the 60 
Wall again in box 278. If A has already reached its maxi- 
mum value, no further stretching is possible, so the Wall 
is presented in box 278. Then, the loop begins again 
with the step in box 270, possibly after an appropriate 
delay so that animated steps are evenly timed. 65 

If the user is depressing a key that requests a shrink 
operation, the step in box 280 tests whether the stored 
value A is already at the value zero. If so, no further 
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shrinking is possible, and the Wall is presented in box 
278. But if not, A is decreased by one incremental 
amount in box 282 before presenting the Wall in box 
278. 

The incremental amounts in boxes 286 and 292 should 
be sufficiently small that the animation steps produce 
the perception that the stretched or shrunken Wall is a 
continuation of the previously presented Wall and to 
preserve object constancy for node display objects. 

If the user has requested scrolling, the step in box 284 
branches based on whether a node display object was 
selected or not. If a node display object was selected, 
the step in box 286 finds the scroll step length and sets 
the necessary number of scroll steps as described above 
in relation to FIG. 6. Otherwise, continuing scrolling is 
requested, so that the step in box 288 determines a scroll 
step length as described above and sets the number of 
scroll steps to one. 

If no user request was made, as determined in box 
270, the test in box 290 determines whether any scroll 
steps remain to be made, which can occur if the step in 
box 286 sets a number of scroll steps greater than one. If 
so, or after one of the steps in boxes 286 and 288, the 
step in box 292 scrolls the Wall by moving it one step 
length by modifying the values LW and RW, discussed 
above in relation to FIG. 3. If LW is decreased and RW 
is increased, the Wall steps rightward, but if LW is 
increased and RW is decreased, the Wall steps leftward. 
The number of steps is also decremented in box 292. 
Then, the Wall is again presented in box 278. 

If the user has made another request, the request is 
handled as appropriate in box 294 before presenting the 
Wall again in box 278. An example of another request is 
a request for a search operation. In response to a request 
for a search operation, a search is performed for nodes 
or files that match the search parameters, and only those 
nodes are displayed, providing a progress indicator of 
the search. Another example is a request for an opera- 
tion setting up the Wall prior to its first presentation. In 
response to a wall setup request, initialization is per- 
formed so that the Wall is appropriately presented in 
box 278. 

The implementation could use the animation tech- 
niques described in Robertson, G. G., Card, S. K., and 
Mackinlay, J. D., 'The Cognitive Coprocessor Archi- 
tecture for Interactive User Interfaces," Proceedings of 
the ACM SIGGRAPH Symposium on User Interface 
Software and Technology, Williamsburg, Va., Nov. 
13-15, 1989, pp. 10-18, incorporated herein by refer- 
ence. 

4. The System 

FIG. 8 shows components of system 300 implement- 
ing the features described above. System 300 includes 
processor 302, which could be any processor, including 
one or more CPUs, and may include specialized hard- 
ware such as one or more coprocessors to provide spe- 
cific functions. Processor 302 is connected for receiving 
signals based on user input actions from a user input 
device that includes keyboard and mouse 304, and is 
also connected for providing image data to an image 
output device that includes display 306. Processor 302 is 
also connected for accessing program memory 310 and 
data memory 330. 

Program memory 310 includes instructions executed 
by processor 302 during its operation, including the 
following: Underlying system software and firmware 
312 provide the operating system and other facilities of 
system 300. Wall setup subroutine 314 initializes values 
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necessary for initial . presentation of the Wall, as de- 
scribed in relation to FIG. 7. Presentation subroutine 
316 presents the Perspective Wall, as described in rela- 
tion to FIG. 4. Selection subroutine 318 is called to 
handle a user selection of a node on the Perspective 5 
Wall, and sets up a scrolling operation as described in 
relation to FIG. 6. Search subroutine 320 performs a 
search for nodes meeting specified parameters. Anima- 
tion loop routine 322 is executed continuously, as de- 
scribed above in relation to FIG. 7, and makes calls to 10 
the subroutines described above when appropriate. 

Data memory 330 includes data structures accessed 
by processor 302 during execution of instructions in 
program memory 310, including the following: Data- 
base 332 is an underlying data structure accessed by 15 
main wall routine 314, such as a set of Unix files. Node 
data structure 334 is produced by main wall routine 314, 
and includes a data item for each node that has a display 
object presented on the Wall; node data structure 334 is 
accessed by presentation subroutine 316 and other sub- 
routines. Database 332 can thus have any form that 
admits of the production of a data structure of nodes 
relating to its contents. Miscellaneous data 336 includes 
initial values and other data used in executing instruc- ^ 
tions in program memory 310, and can include values of 
coordinates of one of the edges of the Wall in three-di- 
mensional space, LW, W, RW, A, 0, the number of 
remaining scroll steps, the length of each scroll step, 
and the size of each incremental amount by which A is 3Q 
increased or decreased. 

D. Miscellaneous 

The invention has been described in relation to a 
workspace with a middle section and with peripheral 
sections on the left and right of the middle section, but 35 
the peripheral sections could be on the top and bottom 
of the middle section, and peripheral sections could be 
provided on all four sides of the . middle section. 

The invention has been described in relation to a 
workspace containing card-like display objects and ^ 
bar-like display objects, but might also be applied to 
other workspaces, such as a workspace within which a 
document is presented, to provide greater detail about a 
part of the document's text, graphics, or other contents. 
The invention has been described in relation to a contin- 45 
uous workspace within which the card-like display 
objects and bar-like display objects could occupy any 
position, but might also be implemented for a work- 
space in which such display objects can occupy a num- 
ber of discrete positions. The invention has also been 50 
described in terms of uniform stretching over a section 
of a wall, but the stretching need not be uniform. Fur- 
thermore, in the case of glued display objects, the inven- 
tion has been described in terms of continuous expan- 
sion during stretching, but a discrete change in a display 55 
object could occur at a given degree of stretching, such 
as through cartographic scaling. 

Although the invention has been described in relation 
to various implementations, together with modifica- 
tions, variations and extensions thereof, other imple- 60 
mentations, modifications, variations and extensions are 
within the scope of the invention. The invention is 
therefore not limited by the description contained 
herein or by the drawings, but only by the claims. 

What is claimed is: 65 

1. A method of operating a processor connected for 
providing image data to an image output device that 
includes a display, the image output device presenting 
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images on the display in response to the image data, the 
method comprising acts of; 
providing first image data so that the image output 
device presents a first image, the first image includ- 
ing a first set of two or more display features that 
are perceptible as positioned relative to each other 
in a fust workspace; and 
providing second image data so that the image output 
device presents a second image following the first 
image in sequence, the second image including a 
second set of two or more display features that are 
perceptible as continuations of the first set of dis- 
play features, the second set of display features 
being perceptible as positioned relative to each 
other in a second workspace; 
the first and second workspaces being perceptible as 
surfaces; each surface having first and second sec- 
tions that meet at an angle along a first edge so that 
first and second sections are perceptible in three 
dimensions; the angle not being equal to 180 de- 
grees; the first and second sections of the second 
workspace being perceptible as continuations re- 
spectively of the first and second sections of the 
first workspace; 
the second workspace being perceptible as a 
stretched continuation of the first workspace; the 
second workspace including a stretched part in its 
first section that is perceptible as a stretched con- 
tinuation of a part of the first workspace; the sec- 
ond workspace also including a part in its second 
section that accommodates stretching. 

2. The method of claim 1 in which the first set of 
display features includes a first display object that is 
perceptible as positioned in the part of the first work- 
space and the second set of display features includes a 
second display object that is perceptible as a continua- 
tion of the first display object and as positioned in the 
stretched part of the second workspace, the second 
display object being the same size as the first display 
object 

3. The method of claim 1 in which the first set of 
display features includes a first display object that is 
perceptible as positioned in the part of the first work- 
space and the second set of display features includes a 
second display object that is perceptible as a stretched 
continuation of the first display object and as positioned 
in the stretched part of the second workspace. 

4. The method of claim 3 in which the second display 
object includes internal detail not included in the first 
display object 

5. The method of claim 3 in which the second set of 
display features includes a new display object that is not 
perceptible as a continuation of any of the first set of 
display features, the new display object being percepti- 
ble as positioned in the stretched part of the second 
workspace. 

6. The method of claim 1 in which the first set of 
display features includes a first display feature that is 
perceptible as positioned in the first section of the first 
workspace and the second set of display features in- 
cludes a second display feature that is perceptible as 
positioned in the second section of the second work- 
space, the second display feature being perceptible as a 
continuation of the first display feature through move- 
ment of the first display feature from the first section 
across the first edge to the second section. 

7. The method of claim 6 in which the first and sec- 
ond display features are display objects. 
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8. The method of claim 1 in which the first and sec- 
ond workspaces each further include respective third 
sections that meet the respective first sections along a 
respective second edge, the third section and second 
edge of the second workspace being perceptible as con- 5 
tinuations respectively of the third section and second 
edge of the first workspace; the third section of the 
second workspace being perceptible as modified from 
the third section of the first workspace to accommodate 
stretching of the streched part 10 

9. The method of claim 1 in which the display has a 
screen surface viewed by a user, the first and second 
sections in each of the first and second workspaces 
being perceptible as two-dimensional surfaces with the 
first section of each of the first and second workspaces IS 
being in a plane substantially parallel to the screen sur- 
face and the second section of each of the first and 
second workspaces extending from the first edge in a 
direction away from the user. 

10. The method of claim 9 in which the angle is ap- 20 
proximately 135 degrees so that the second section of 
each of the first and second workspaces occupies rela- 
tively little of the screen surface, 

11. The method of claim 1 in which the first set of 
display features includes a first display object and the 25 
second set of display features includes a second display 
object, the second display object being perceptible as a 
continuation of the first display object through move- 
ment of the first display object 

12. A method of operating a processor connected for 30 
providing image data to an image output device that 
includes a display, the image output device presenting 
images on the display in response to the image data, the 
method comprising two or more acts performed in 
sequence, each act comprising: 35 

providing image data so that the image output device 
presents an image, each image including a set of 
two or more display features that are perceptible as 
positioned relative to each other in a workspace; 

the sequence of acts including a preceding act and a 40 
following act that follows the preceding act in the 
sequence; the set of display features of the follow- 
ing act being perceptible as continuations of the set 
of display features of the preceding act; 

the respective workspaces of the preceding and fol- 45 
lowing acts being perceptible as surfaces; each 
surface having first and second sections that meet 
at an angle along a first edge so that first and sec- 
ond sections are perceptible in three dimensions; 
the angle not being equal to 180 degrees; the first 50 
and second sections of the following act being 
perceptible as continuations respectively of the 
first and second sections of the preceding act; 

the first section of the following act including a 
stretched part that is perceptible as a stretched 55 
continuation of a part of the first section of the 
preceding act; the second section of the following 
act also including a part that accommodates 
stretching. 

13. The method of claim 12 in which the processor is 60 
further connected for receiving signals from a user 
input device, the sequence of acts including a first act; 
the method further comprising receiving signals from 
the user input device requesting presentation of an in- 
creasingly stretched version of the workspace of the 65 
first act; the respective workspaces of the preceding and 
following acts being perceptible as increasingly 
stretched continuations of the workspace of the first act 
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in a sequence of stretched continuations leading from 
the workspace of the first act to the workspace of the 
preceding act and thence to the workspace of the fol- 
lowing act. 

14. The method of claim 12 in which the set of display 
features of the preceding act includes a first display 
feature that is perceptible as positioned in the first sec- 
tion of the workspace of the preceding act and the set of 
display features of the following act includes a second 
display feature that is perceptible as positioned in the 
second section of the workspace of the following act, 
the second display feature being perceptible as a contin- 
uation of the first display feature through movement of 
the first display feature from the first section across the 
first edge to the second section. 

15. A system comprising: 

an image output device that includes a display; and 
a processor connected for providing image data to 
the image output device; the image output device 
presenting images on the display in response to the 
image data; 

the processor operating to provide the image data so 
that the image output device presents a sequence of 
images, each image including a set of two or more 
display features that are perceptible as positioned 
relative to each other in a workspace; 

the sequence of images including a preceding image 
and a following image that follows the preceding 
image in the sequence; the set of display features of 
the following image being perceptible as continua- 
tions of the set of display features of the preceding 
image; 

the workspaces of the preceding and following im- 
ages being perceptible as surfaces; each surface 
having first and second sections that meet at an 
angle along a first edge so that first and second 
sections are perceptible in three dimensions; the 
angle not being equal to 180 degrees; the first and 
second sections of the following image being per- 
ceptible as continuations respectively of the first 
and second of the preceding image; 

the first section of the following image including a 
stretched part that is perceptible as a stretched 
continuation of a part of the first section of the 
preceding image; the second section of the follow- 
ing image also including a part that accommodates 
stretching. 

16. The system of claim 15, further comprising a user 
input device, the processor further being connected for 
receiving signals from the user input device; the se- 
quence of images including a first image; the processor 
further operating to receive signals from the user input 
device requesting presentation of increasingly stretched 
versions of the workspace of the first image; the respec- 
tive workspaces of the preceding and following images 
being perceptible as increasingly stretched continua- 
tions of the workspace of the first image in a sequence of 
stretched continuations leading from the workspace of 
the first image to the workspace of the preceding image 
and thence to the workspace of the following image. 

17. The system of claim 15 in which the set of display 
features of each of the images includes a set of display 
objects, the system further comprising a data structure 
that includes a set of two or more items, each item 
including data for a respective one of the set of display 
objects in each of the images; the processor further 
being connected for accessing the data structure in 
providing the image data. 
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18. A method of operating a processor connected for 
providing image data to an image output device that 
includes a display, the image output device presenting 
images on the display in response to the image data, the 
method comprising: . 5 
providing first image data so that the image output 
device presents a first image, the first image includ- 
ing a first set of two or more display objects that 
are perceptible as positioned relative to each other 
in a first workspace; each of the first set of display 10 
objects having extent and being bounded; and 
providing second image data so that the image output 
device presents a second image following the first 
image in sequence, the second image including a 
second set of display objects that are perceptible as 15 
continuations of the first set of display objects, the 
second set of display objects being perceptible as 
positioned relative to each other in a second work- 
space; each of the second set of display objects 
having extent and being bounded; 20 
the first and second workspaces being perceptible as 
surfaces, each surface having first and second sec- 
tions that meet at an angle along a respective first 
edge so that the first and second sections are per- 
ceptible in three dimensions; the angle not being 25 



equal to 180 degrees; the first section, second sec- 
tion, and first edge of the second workspace being 
perceptible as continuations respectively of the 
first section, second section, and first edge of the 
first workspace; 
the second workspace being perceptible as a 
stretched continuation of the first workspace; the 
second workspace including a stretched part in its 
first section that is perceptible as a stretched con- 
tinuation of a part of the first workspace; the sec- 
ond workspace also including a part in its second 
section that accommodates stretching of the 
stretching part in its first section. 

19. The method of claim 18 in which each display 
object in the first and second set of display objects is 
card-like. 

20. The method of claim 19 in which each display 
object has four sides. 

21. The method of claim 18 in which each display 
object in the first and second set of display objects is 
bar-like. 

22. The method of claim 18 in which the angle is 
approximately 135 degrees. 
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